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1  Introduction 


W (' fst aMisli  a  nt'iicral  r<'siill  alxmi  alucluaic  t 

acros,''  ihrcf  |)i  rs|iri  ti\i-  \  ii  wx  of  a  1(1)  m  i'iii'  ami  (li  iiinn- 
slrali'  its  a|i|)lirai  ion  to\  isnal  ri'cojiiiil  ion  via  alinniiionl . 
Wi'sliow  tlial,  in  u,t  ni‘ral.  any  iIii't  |n‘rs|)t  (  t  ivi-  viows  of 
a  sccno  satisfy  a  pair  of  Irilim  ar  functions  of  <<>- 

ordiimti's.  In  tlic  limit  inn  cas(.  'vlnn  all  ilircc  viivvs  arc 
orl lioni't'pl'i'  -  >licsc  fnnciions  hcconic  linear  ami  rcilncc 
to  tile  form  iliscoveieil  by  [ifl],  I  sinji  tile  triliiiear  result 
one  can  manipniati'  \  lews  of  an  ob  ject  (such  .as  nem  raie 
novel  views  from  two  nuxlel  views)  without  recoverinn 
scene  slrnclnre  (metric  or  non-metric),  camera  Ir.tnsfor- 
mation.  or  even  the  epipolar  .neometry. 

The  central  resnlls  in  this  paper  :ire  contained  m  1  lie- 
ort'ins  1  ami  2.  1  he  tirsi  theorem  stales  that  the  vari¬ 

ety  of  views  (,  ol  a  li.xed  21)  object  obi, aim'd  by  an  nn- 
c.alibraled  pin-hole  camera  satisfy  ,a  relation  of  the  sort 
/  ( i  .  I.  1 ,  i-j )  =  0.  where  i.  i .  i -j  are  two  arbiir.arv  views 
of  the  object,  and  /'  has  .a  special  Irilimar  form.  Ilie 
coefficients  of  /'  ran  be  recovered  linearly  withoni  es- 
tablishinu,  first  the  epipolar  geometry.  21)  slrnclnre  of 
the  object,  or  camer.a  motion.  The  an.vciliary  Li'iimias 
reipiired  for  the  proof  of  Theorem  1  may  be  of  interest 
on  their  own  ;is  they  esiafilish  la-rlain  regnlarities  across 
lirojective  transformations  of  tin'  plane  and  inlroiiiice 
new  view  invariants  (Lemma  f). 

Theorem  2  is  an  obvious  corollary  of  'Theorem  1  but 
contains  a  significant  practical  as|>('cl.  It  is  shown  that 
if  tlie  views  t.  i.t.  -j  are  obtained  by  parallel  projection, 
then  /'  reduces  to  a  s|)<'cial  bilinear  form  or,  e(|uiva- 
lently.  that  any  perspective  view  f  can  be  obiaim'd  by  a 
rational  linear  function  of  two  orthographic  views.  The 
reduction  to  a  bilinear  form  implies  that  simpler  recog¬ 
nition  schemes  are  possible  if  the  two  reference  views 
(model  views)  stored  in  memory  are  orthographic. 

Tht'se  two  results  may  have  si  veral  ajiplicat ions  (di.s- 
enssed  in  Section  (i).  but  the  one  emplnisizevl  thronghont 
this  pajier  is  for  the  task  of  recognition  of  21)  objects 
via  alignment.  The  alignment  apjiroach  for  recognition 
([22,  16],  and  references  therein)  is  based  on  the  result 
that  the  e(|ni valence  class  of  vii'ws  of  an  object  (ignor¬ 
ing  self  occlusions)  nndergoing  21)  rigid,  affine  or  pro¬ 
jective  transformations  can  be  ca|)tnred  by  storing  :i  21) 
model  of  the  object,  or  simiily  by  storinj;  at  least  two 
arbitrary  "modeT'  views  of  I  he  object  assuming  that 
the  corre.s|)ondence  [vroblem  between  the  model  views 
can  somehow  be  solved  (cf.  ['i').  o.  29]),  During  recog¬ 
nition  a  small  number  of  corresponding  |)oints  between 
the  novel  input  view  and  the  model  views  of  ;i  particular 
candidate  object  are  sufficient  to  "re  project"  the  model 
onto  the  iifjvel  viewing  position.  Recognition  is  achieved 
if  the  re-projected  image  is  snccessfnlly  matched  against 
the  in])nt  image.  We  refer  to  the  problem  of  |)redirtiiig 
a  novel  view  from  a  set  of  model  views  using  a  limited 
nnmlier  of  corresponding  |)ointx,  as  the  problem  of  n- 
proj((  lion. 

The  problem  of  re-[>rojertif)n  can  in  jrrincipal  be  dealt 
with  via  21)  reconstruction  of  shape  and  camera  motion. 
This  includes  classical  structure  from  motion  methods 
for  recovering  rigid  camera  motion  parameters  and  met¬ 
ric  shape  [22.  If'.  21.  I  I.  la],  and  more  recent  meth- 


i)d>  lor  recoseriny,  non-meini  xtrncinre.  i  ,  ..  avsnmmg 
th  ■  objects  nnderi!,o  21)  .tfline  or  projiciui  ir.insforma- 
lions.  or  equivalently,  that  the  i  ;mieras  an  imcalibraled 
[IT.  22.  2T,  It).  12.  27,  2s].  The  rlassiv  approa.hes  for 
perspective  \ii  ws  are  known  lo  be  unstable  under  errors 
in  im.age  measurements,  narrow  field  of  view  .  and  inier- 
n;d  camera  calibration  [2.  9,  12],  and  therefore,  an  nn 
likely  to  be  of  |>racl  leal  use  lor  purposes  ol  re-pro  |eet  Ion . 
file  non-meirie  apjiroaches.  as  a  general  concepi.  h.ave 
not  been  fully  listed  on  real  imayes.  but  the  methods 
preiposed  so  far  rely  on  reeoveriny  first  the  epipolar  ye- 
oiiietry  a  pro<-ess  that  is  also  known  to  be  unstable  in 
t  he  presence  ol  noise. 

it  is  also  known  t  li.it  t  he  epipolar  yeoim  t  ry  is  by  itself 
snflicient  to  achieve  re-project  ion  by  means  of  inteiseei- 
iny  epi|)olar  lines  [22.  li.  .s;,  21.  21,  11].  This,  however, 
is  |iossib|e  only  il  the  cenlers  ol  the  three  cameras  are 
iion-colline.ir  which  can  lead  to  nnmerical  instability 
unless  the  centers  are  far  from  colliiiear  and  any  ob¬ 
ject  point  on  the  tri-lbcal  plane  cannot  be  re-projer|ed 
as  well,  Tnrt hermore.  as  with  the  non-niei ric  reconst rnc- 
t  ion  met  hods,  obt  aining  t  he  epipolar  yeomei  ry  is  at  best 
a  sensitive  process  even  when  dozens  Ol  corresponding 
points  an-  used  and  with  the  stale  of  the  art  methods 
(see  .Section  a  for  more  details  aiul  comparative  analysis 
with  simulated  and  real  images). 

For  pnrpo.ses  of  stability,  therefore,  it  is  worthwhile 
e.\|>loring  more  direct  tools  for  achieving  re-projection. 
For  instance,  instearl  of  ri'constrnclion  of  shajie  ami  in¬ 
variants  we  wcnild  like  to  establish  a  direct  connection 
between  views  expressed  as  a  functions  of  image  cot)r- 
dinates  alone  which  we  will  call  "algebraic  functions 
of  views".  Snell  a  result  was  established  in  I  lit'  ortho- 
graphie  ca.se  by  [21].  There  il  was  shown  that  any  ihrt'e 
orthographic  views  of  an  object  satisfy  a  linear  function 
of  the  corresponding  image  coordinates  this  we  will 
show  here  is  simply  .a  limiting  case  of  larger  set  of  al- 
gebraic  functions,  that  in  gi'iieral  have  a  triliiiear  form. 
With  the.se  fnnciions  one  can  manipulate  views  of  an 
object,  such  as  create  new  views,  without  the  need  to 
recover  shape  or  camera  geometry  as  an  intermediali- 
step  all  what  is  m-.-di'd  is  to  appropriately  ci'inbine 
the  image  coordinates  of  two  reference  views.  Also,  with 
these  fnnciions,  the  epipolar  geometries  are  intertwined, 
leading  not  only  to  absence  of  singularities,  but  as  we 
shall  se<'  in  the  experimental  section  to  more  accurate 
performance  in  the  presence  of  errors  in  image  measure¬ 
ments. 

2  Notations 

W  e  consider  object  space-  to  be-  the  t hree'-dimi'iisional 
proje'Clive  space  'P'\  and  image  space  to  be  the  Iwo- 
elillK'lisional  projective'  space  P- .  l.e  l  <l>  C  T'*  be'  a  se't  eef 
points  staiieling  fe)r  a  21)  object,  anel  h't  i.  ,  C  P'  eli'iieete- 
views  (arbitrary),  ineh'xe'el  by  i.  ejf  <I).  (live'ii  two  e  am- 
eras  with  centers  lejcate-el  at  ().()'  €  P’.  re'spe'cl ive'ly.  the- 
e'pipoh'S  are-  ele'fim'el  te)  be'  at  the'  ini e'r.se'Cl iejll  of  the'  line' 
()()'  with  both  image-  plane's.  Bi'cansi'  the'  image'  plane'  is 
finite',  we  can  assign,  witlmiit  hiss  e)f  ge-ne-ralily.  the-  value' 
I  as  the'  thirel  he)me)ge'neons  ceieirelinale'  tei  e-ve-ry  obsinul 
image' point.  I  hat  is.  if(j-,  //)  are-  t he- obse-rve-el  image- eee- 


onliiiatt's  of  st)iiU' |>i)iii[  («  i(li  rcspcci  to  siiim- arliiliarv 
orif^iii  say  lln'  ‘'■ntt'r  ot  tlu"  iiua"t‘),  tln'ii 

l>  =  (.)•,,(/,  1)  tliMH)tt's  tlif  lioiiionfiifoiis  i-otiialiiiatos  of 
tile  iiiiagf  plaiir.  Siiicr  \vc  will  l)f  woikiii;^  «itli  at  most 
tlirt't'  vii'ws  at  a  time,  we  deiioti'  the  relevant  e|)i|K)|es 
as  follows;  let  i  t  i  i  ainl  r'  t  i  j  he  the  eorrespoinliii;; 
epipoles  heiween  V  ii'ws  r|.  l  -j.  ami  let  i  €  I’l  atid  t  "  t 
la  'III*  cot  respoiulilif^  epipoles  hetweeil  vit'Ws  l  i  .  l 
Likewise,  coirespoiuliiiff  ima};e  points  across  three  views 
will  he  ih'lioteti  hy  ft  =  ( .c , ,(/,  1  ) , />'  =  (.r'.i/,l)  and 
ji"  =  (j",  ,!/".  1).  I'he  t('rm  '  imane  coordinates"  will  de¬ 
note  the  non-homofi,eneoiis  coordinate  rf'presentation  of 
P- .  e,>;,,  ij).  (.r',  (j",  //"  )  for  the  three  Correspond¬ 

ing;  points. 

Planes  will  he  denoted  hy  t,  ,  indexed  hy  i.  and  just  t 
if  only  OIK'  plant'  is  discussed.  .\11  plant's  are  a.ssnmt'd  to 
he  arbitrary  and  tiistinci  from  one  another.  1  In'  symbol 
=  tlenott's  ('quality  up  to  a  scale.  (iL„  st;mds  for  the 
Uruup  of  n  x  n  matrices,  aiul  Pdl.,,  is  the  jtroiip  dt'fined 
up  to  a  scale. 

A  coordinate  -t'presentatioii  R  of  P*  is  a  tetrad  (,f 
coordinates  ;i.  zj.  la]  such  that  if  Tvn  is  any  one  al¬ 
lowable  represt'iitation.  tin*  whoh'  class  R  consists  of  all 
I  host'  rt'iirt'st'iilations  that  can  he  obtained  from  7\n  by 
tht'  action  of  the  group  P(lLi.  (iivt'u  a  set  of  views  i-,. 

i  =  1.2 . of  <1>.  where  coordinates  on  iq  are  [j-.  y.  1]  ami 

Ru  is  a  reiirest'iitatioii  for  which  =  (j-.y.  1). 

we  will  say  that  the  ohjt'ct  is  uiuh'rgoing  at  most  dD 
r(tatn'(  affine  transformations  between  vit'ws  if  tin'  class 
of  rt'presentations  R  consists  of  all  those  representations 
that  can  he  obtained  from  7v(i  by  the  action  of  an  affiiu 
subgroup  of  P(i L.\.  Ill  other  words,  tin'  object  undergoes 
some  projective  traiisformation  and  projected  onto  the 
view  I'l.  aftt'f  which  all  other  transformations  applied  to 
<I>  are  affine.  .Note  that  this  definition  is  genera!  and  al¬ 
lows  full  uncalilirated  piii-hoh'  camera  motion  (for  more 
details  on  uncalibrated  camera  motion  versus  relative 
affint'  transformation  versus  taking  pictures  of  pictures 
of  the  scene,  see  Ap|)endix  of  [2()]). 

3  The  Trilinear  Form 

riit'  central  result  of  this  paper  is  (iresentt'd  in  the  fol¬ 
lowing  theorem.  The  remaining  of  the  section  is  devoted 
to  the  proof  of  this  result  and  its  implications. 

Theorem  1  (Trilinearity)  Ld  iq.iv.t.a  tx  thr((  ar- 
biivari)  p(  vsjH dux  i  n  ws  of  soiiu  olijid.  iiioddfd  bif  a  sd 
of  ponils  III  W.  iiiidf  rgoiini  <il  iiiont  ii  il)  rdatiix  nffiin 
IraiifyfoniKilioiifi  bdiixui  i  n  u  s.  Tin  niingi  roordiiialf  s 
(j'-U)  €  I'l-  (•'■'•.'/)  €  IX  (uid  (j".y")  £  f3  of  llini 
( orif  spoiidiinj  points  across  tlird  liius  satisfy  a  pair  of 
Iriinnar  (qiialioiis  of  tin  folloirnig  form: 

J-"(o  l.r  -t-  o-jy  +  03)  -I-  +  tir.y  +  (i,;)-|- 

-b  Oft  1/  -j-  o  )  -4-  o  I fiJ'  4-  o  1 1 .1/  -b  o  i i>  =  0. 

and 

-b  fii.y  -b  fa)  -b  fl  -T  ■/-,(/  -b  -fi  j-b 

x'{  'ijj'  -b  ds.y  -b  dll )  -b  b  c-f  +  dj  1  y  -b  .f  1 3  =  t). 


ir/n  ri  t/n  <  o<  jjn  n  ills  ti , .  j  =  I .  12.  an  ju  td  joi 

all  points,  an  aiinjiidg  ddund  up  to  an  on  rail  si  ah  . 
and  i\j  =  t , .  /  -=  1 . (). 

Ml  f  follow  iiiu,  ,tuxiliar\  j)ro|iosii  ion>  an  usi  ti  .1.-.  p.u  i  of 
the  proof 

Lemma  1  (Auxiliary  -  Existeiici  )  id  .1  b  P( d.n 
In  till  piojidiri  nnippiag  I  lioinogra  pli  g  I  iq  > —  1  ■.  dm  to 
soon  plain  J  l.d  .1  In  sialnl  In  salisjg  p'  =  .\p..  -b  n' 
irin  r<  p..  £  iq  and  p' .  £  i  ■,  an  l  orn  sponding  points 
coining  from  an  arbitrarg  point  /',  g  rr.  Thin,  for  nng 
corn  sponding  pan  p  £  iq  and  p'  £  1  l  oiiiing  jroiii  an 
arbitral  g  point  P  £  P‘f  in  Inin 

p'  =  .\p  +  kv 

I  In  cm  )]n  n  III  k-  is  unit  pi  ndi  nl  oj  ii.  /  1  .  is  iiiniriani 
to  tin  ilioiii  oj  tin  stniiid  run. 

file  lemma,  its  proof  and  its  tht'oi'elical  ami  practical 
implical ions  are  ilisciissed  in  detiiil  in  [2t)].  Note  that 
the  p.'irlicular  case  where  the  homography  .1  is  atline, 
and  the  epipole  ('  is  on  the  line  at  inlinitv.  ctirresitomls 
to  the  construction  of  affiiit'  structure  from  two  ortho¬ 
graphic  view's  [17].  I  he  scalar  b  is  calh'd  a  ri  latiri  ajfiin 
inniriaiil  and  represt'iits  the  ratio  of  tin'  dislanct-  of  P 
from  T  along  the  line  of  sight,  and  tin'  ilisiaiice  of  P 
from  the  camt'ra  ceult'r  of  I'l.  normalizi'd  by  the  ratio 
of  disianct's  of  P,,  from  the  plane  and  the  camera  center, 
riiis  normalizi'd  ratio  ran  he  com|>uted  with  the  aid  of 
a  second  arbitrary  view  i'_,. 

Definition  1  Homographns  .1,  £  PdL.i  from  I'l  »—  i-, 
dm  to  lilt  saint  plant  rr.  art  said  to  In  st  all  -nnnpalibli 
if  liny  art  staltd  to  satisfy  Ltnimti  I.  i.t..  for  any  point 
P  £  <t>  projtdiny  onto  p  £  iq  and  p'  £  L  j.  tinrt  tnsls  a 
scalar  b  that  salisjits 

p'  S  .\ip+  bv' . 

for  any  rntr  i', .  ii  jitri  r'  £  i  ,  is  tin  tpipoh  intli  iq 
(scaldl  arbilrarityj. 

Lemma  2  (Auxiliary  Uniqueness)  l.tt  .1..1'  £ 

PdL.i  liomograpliit s  e/ iq  i —  I'v  dm  to  plants 

TTi.Tq.  rtsptdinly.  Thtii.  Ilurt  crisis  a  stiiltirs.  thal 
satisfies  tin  (qiiation: 

.1  —  S.\'  =  [old.  .hd.  -  id], 

for  soiin  cot  fficit  nis  o  .  .i.  . 

Proof.  Lt't  q  £  I'j  he  any  point  in  the  first  view. 
There  t'xists  a  scalar  .s,,  that  satisfies  i'  =  .h/  —  s^.\'q. 
[a't  //  =  .\  —  s,^A'.  and  we  havt'  Hq  =  But.  as  shown 
in  [27].  .If  =  r'  for  any  homogra|)hy  iq  i —  fv  due  to  any 
plane.  Therefore.  Hr  —  r'  as  well.  The  mapping  of  two 
distinct  jioints  q.  r  onto  the  same  point  c'  could  happt'ii 
only  if  // p  S  r'  for  all  yi  £  iq  .  and  .s^  is  a  fixed  scalar  .s. 
This,  in  turn,  im|dies  that  //  is  a  matrix  whose  columns 
are  multiiiles  of  v' .  [] 

Lemma  3  (Auxiliary  for  Lemma  4)  Lt  t  .1.1'  £ 

PdL.i  lx  homogrtiphn s  from  iq  i —  i  j  dm  to  distinct 
plain  rri.rrq.  rtsptdinly.  and  H.B'  £  Pdlm  bt  lionio- 
graplins  from  L]  t—  r.q  dm  /eT|.;r>.  rtsptdinly.  L'htn. 
A'  =  AT  for  soint  T  £  PdL.i.  oiid  B  =  BdT(-'. 

I  win  I  t  (  'r  ~  r 


Proof.  Lfl  .1  =  .1.7'. li,  will'll'  an-  lioiiin- 

jirapliies  from  t  1 .  i  j  onto  tti.  ri's|it  rii\i'ly  Similarl> 
li  =  li~  '  li^ ,  wlifi'i'  Hi  .  Hj  all'  liomoi;ra|ilili'N  from  t  | ,  i 
onto  Ti  ,  resju'Cl  ivciy.  Lrl  .Iji  =  (I'l  ,  r;j)^  .  ainl  li-l 

('  =  .Ij" '(//(/l/(l•l  c:)).!  I .  fill'll.  H[  .Ilf'""',  iiiil 

liiiis,  wt'  liavi'  H  ~  H~'  .  .Noll-  llial  lln' only  ilif- 

ft'li'lict'  bi'lwi'i'ii  .li  ami  H[  is  dm  to  tin'  dllfi'rilit  lo¬ 
cation  of  till'  i'|)i[ioli's  (  .r.  uliicli  is  I'oiiiin'iisatcd  l>y  (' 
((  '1  S  I  ).  fi’l  /.  I  G  HCil-w  III'  till'  liomoi;ra|diy  from  1.1 
to  T_),  and  fc  HdL.s  the  liomo!;ra|iliy  from  t-j  to  n-i. 
fill'll  with  |iro|ii'r  scaliii;;,  of  /•.  1  and  /.'•_>  wi-  liav*- 

.1'  =  .i7‘/.'j/.'i  =  .1.17 =  .17', 

and  with  iiropi'r  scalinj;  of  (  '  wc  havi', 

H'  =  Bf '  A',  f  -  ‘  =  7yr.  1 7 '  i:-,  /;,  r- '  =  /;f  ■  /  y  -  ‘ . 

D 

Leiiiiiia  4  (Auxiliary  Uuitiuoiiess) 

Foi  s(  tilt -I  (imiKtlililf  lioiiiot/riiiiliii  s.  lilt  si  tilfirss.ti.  i.- 
of  Iaiiiiiiii  J  an  niiariaiil.'<  indt.rttl  hi/  Thai 

IS.  gin  II  an  arbilrarg  Ihiid  ritii-  l'a.  It  I  B.  B'  ht  tin  lio- 
niogiai)hii  s  from  I'l  1 —  i';;  din  to  Ti.ir-j.  rtsinflinlg.  Lit 
B  III  siiilf -I oiiiiialihli  irilli  .1.  and  B'  ht  snilt-toiiipalililt 
iritli  .1'.  TInn. 

B-sB'  =  [nv''..iv".-,v‘']. 

Proof.  WV  show  lirst  that  .s  is  invariant,  i.e..  that  B  — 
■sB'  is  a  matrix  whose  colnmtis  are  tiinlt iples  of  f".  from 
Lemma  '2.  and  Lemma  d  there  exists  a  tnatrix  //.  whose 
colnmns  are  multi))les  of  r' .  a  matrix  7  that  satisfies 
.-V  =  .17',  and  a  scalar  .s  such  that  I  —FT  =  .1”'  //.  .After 
tmtltiplying  both  sides  by  BC.  and  theti  pre-tnnltiplying 
l>y  (  '  we  obtain 

B-sB(T(-^  = 

Frotii  Lemma  d,  we  have  B'  =  B('T('~^.  The  ma¬ 
trix  has  colnmns  which  are  multiples  of  r  (be¬ 

cause  .l“'i'  =  I'),  ('A~^n  is  a  matrix  whose  colnmns 
are  multiple  of  r.  and  B('A~^H  is  a  matrix  whose 
columns  are  multiples  of  r" .  Pre-multiplying  777 '.A” ' // 
by  7'“'  does  not  change  its  form  because  every  colninn 
of  B('.\~^  H('~^  is  simjily  a  linear  combination  of  the 
colnmns  of  B('.\~^H.  As  a  result.  B  —  .s/7'  is  a  matrix 
whose  colnmns  are  multiiiles  of  r" . 

Let  H  =  .1  — .s.  l'and  //  =  77— .s/7'.  Since  the  homogra- 
phies  are  scale  com[)atible.  we  have  from  Leinina  1  the 
existence  of  invariants  k.k'  associated  with  an  arbitrary 
ji  €  i-'i.  where  /•  is  due  to  tti.  and  k'  is  due  to  n->:  ]>'  = 
.\pFkv'  ^  ,\'p  +  k'v'  and  /."  S  BpTkv"  S  /7'p-p7'r". 
Then  from  Lemma  2  we  have  lip  =  {.sk'  —  k)v'  and 
Hp  =  (.sk'  —  k)v" .  Since  p  is  arbitrary,  this  could  hap- 
pen  only  if  the  coefficients  of  the  multiples  of  r'  in  7/ 
atid  the  coefficients  of  the  multiples  of  1"  in  77.  coincide. 

D 

Proof  of  Theoroui:  Lemma  1  provides  the  existence 
jiart  of  theorem,  as  follows.  Since  Lemma  1  holds  for 
any  plane,  choose  a  plane  T|  ami  let  ,-1,77  be  the  .scale- 
compatible  honiographies  li'i  I —  aiul  I'l  I—  I'.a.  respec¬ 
tively  Then,  fui  every  point  // G  t'l.  with  corresponihiig 


piiints  p'  G  '  j  G  I.  a.  thin  exist.',  a  scalar  k  that  sat¬ 
isfies:  p'  =  .1/1-1- All',  atid  /'"  =  Bp  +  kv".  Wecaii  isolate 
k  from  both  l■(platlons  atid  obtain: 

it  ~  - - - -rj —  rr  — ■ — * - Tj —  —  - '• — I - !« —  .  I  I  / 

I  I '  it  (I  i  t '  f  1  t, 'it  , -a  J  p  '  . — J  *1  i  I 

^  "  TTTTZTTt  "  rTTrtTTTTTo  ■  '  ' 

where  hj.hj.ln  and  iij.Os.a.)  an  the  row  \eclors  of  .1 
and  77  and  r'  =  (  c' .  i  v.  1 'J.  1"  =  ( 1 .  i7,'.  1  )  tfecau.se 

of  I  he  invariance  of  k  we  can  eipiate  terms  of  l  iipiat  ion  1 
with  terms  of  Kipiatioii  2  atid  obtain  triliin  ar  functions 
of  image  coordinates  across  three  views.  For  exam|ile. 
b\  eipiating  the  lirst  two  teitiis  in  each  of  the  1  ipnitioiis. 
we  obtain: 

.1-  (1  ,t>:i  -  |■.5^I.l  )  ;  .1-  .r  (i.,a:,  -  !  // -(- 

.l•'(|■',hl  -  I'i'a-.i)'  p+  (c'i'mi  -  I'lhi  )'  p  =  b.  id) 

Iti  a  sitiiilar  fashion.  <dter  eipiat  iiig  t  he  first  tertii  of  F.ipta- 
tion  1  with  the  second  tertii  of  Fipiat  ioti  2.  we  obtain: 

.i/'(  i  -  r'.fax  )'  p  +  !i"j'(  r'fiin  -  r'Jn)'  p  -h 

P+  (r"ui  -  r’^b-,)'  p  -  (I.  (1) 

lk>lh  eipiations  are  of  the  desired  fortn.  with  the  first  six 
coefficients  idetitical  across  both  eipiat  iotis. 

1  he  ipiestion  of  ittiiipiene.ss  arises  becaitse  Leitittia  1 
holds  for  atty  platie.  If  we  choose  a  diflereitt  plane,  say 
n-t.  with  hotitographies  .V .  B' .  theti  we  must  show  that 
the  ni'w  homographies  give  rise  to  the  satiie  coefficients 
(up  to  ati  overall  scale).  Fhe  paretithesized  terttis  itt 
Kiptations  d  atid  I  liavr  the  general  form:  r'jb,  ±  r''a.j. 
for  some  /  and  j.  Thus,  we  tieed  to  show  t hat  there  exists 
a  scalar  .s  that  satisfies 

r'/iuj  -  .sa'j)  =  r'(7>,  -  .s7>'). 

This,  however,  follows  directly  from  l.emmas  2  atid  1.  [] 
The  direct  itiijilicat ioti  of  the  theorem  is  that  otie  cati 
generate  a  tiovel  view  (i/a)  by  sitiijily  cottibining  two 
model  views  ('  1 .  (.  •_>).  The  coefficients  atid  of  the 
combination  can  be  recovered  together  as  a  solution  of 
a  linear  system  of  17  eiptatiotis  (21  —  (i  —  1)  given  nitte 
correspotiditlg  jioitits  across  the  three  views  (tiiore  than 
nine  points  cati  be  nsi'd  for  a  least-sipiares  solutiott). 

d'aken  together,  Lipiat iotis  1  and  2  lead  to  9  algebraic 
functions  of  three  views,  six  of  which  are  sejiarate  for  .1" 
and  I/".  The  other  four  functions  are  listed  below: 

•»•"(■) +  .'".i/()  +  .i/()  +  ()  =  0. 

=U. 

r"y(  )  +  .i".i/'(.)-f  =  0. 

■)  +  .t/".'/(')  +  -'  '(  )  +  .i/(  )  =  0, 

where  (•)  represent  linear  polynomials  in  x.g.  Fhe  solu¬ 
tion  for  j".g"  is  niiiipie  without  cotistraints  on  the  al¬ 
lowed  catiiera  transfortiiat iotis.  If  we  choose  Fqnations  d 
and  A,  then  r'j  and  c',  should  not  vanish  simiiltaneonsly. 
i.e.,  ('  =  (U.  1,0)  is  a  singular  case.  Also  1"  =  (0.  1.0) 
and  1"  ~  ( 1 . 0.  0)  give  rise  to  singular  cases.  7)ne  can  eas¬ 
ily  show  that  for  each  singula'’  ca.se  there  ivve  other 
functions  out  of  the  nine  availalde  ones  that  provide  a 


3 


unKiiic  solution  lor  j'',  //".  Nolo  iliat  the  siiif'ular  (•.'iscs 
are  |)oint\visi‘.  i.i  ..  only  llirnc  i'|)i|)olar  dirt  ct ion,',  arc  i-x- 
chultMl,  compari'd  to  thf  inoi  '  wi(l<'-s|uoatl  sinjiulai  rases 
that  oecur  with  e|>i|iolaf  interseetion,  as  descrihetl  in  the 
int  nuhiel  ion. 

In  practical  ternis.  the  process  ot  j^eiieralinj;  a  novi  l 
view  can  he  easily  accomplished  without  the  need  to  ex¬ 
plicitly  recover  St  fuel  uri‘,  catnera  t  ransrormat  ion.  or  just 
the  epipolar  geometry.  I  he  process  descrihed  here  i> 
luiidamentidly  ditl'ereut  I'rom  itttersect iiiji  epipolar  lilies 
in  tin'  lollowing  ways:  first,  wt'  use  the  three  vi<'w>  to- 
get  ln'l'.  instead  ol  inurs  of  views  separately  ;  second,  there 
is  no  process  of  litie  ititersect ion ,  i.e.,  the  j-  and  ,(/  coor- 
ditiates  of  I.  a  tire  obtained  st'pjirately  as  a  solution  ol'  a 
single  ('quatiou  in  coordinates  of  th<‘  other  two  views: 
and  thirdly,  the  process  is  well  defined  in  cases  wln-n- 
ititersect ing  epijiolar  lines  becomes  singular  (e.g.,  when 
the  three  camera  centers  are  collinear).  1- urt  hertiiore.  by 
avoiding  the  need  to  recover  tin-  epipolar  geometry  wi- 
obtain  ;i  significant  jiractical  advantage,  since  the  epipo¬ 
lar  gi'ometry  is  the  most  error-setisit ive  compouetit  when 
working  with  perspective  views. 

1  he  contiectioti  bi'tween  the  general  result  oftriliiiear 
I'utictious  of  views  to  the  "linear  cotnbiuation  of  views' 
result  [d  1]  for  orthographic  views,  can  easily  In'  seen  by 
si'tting  ,1  and  H  to  be  affine  in  P- .  and  I'j  =  r'l  —  0, 
For  exatnple,  L'liuation  d  reduces  to 

-  f"j-'  +  (I'l'a!  -  r\ht)'  i>  =  (). 
which  is  of  the  forttt 

o  I  .f"  +  +  03.J-  -f  o.jg  -p  o.'j  =  0. 

I'hus.  in  the  ca.se  where  all  three  views  are  orthographic, 
j'"  is  ('xpressed  as  a  linear  cotiibitiation  of  image  coordi- 
tiates  of  the  two  other  views  -  as  discovered  by  [dl], 

4  The  Bilinear  Form 

Consider  the  case  for  which  the  two  referenc(>  (model) 
views  of  an  object  are  taken  orthographically  (using  a 
tele  lens  would  provide  a  reasonable  apfiroximation).  but 
duritig  recognitioti  any  |)erspective  view  of  the  object  is 
allowed.  It  cati  easily  be  shown  that  the  three  views  are 
then  contiecteti  via  bilinear  functions  (itislead  of  triliti- 
ear ) : 

Theorem  2  (Bilinearity)  Within  the  (oiiititioiis  of 
Tliforaii  1.  ill  cn.si  1h(  vu  u  s  I'l  aiid  i  -2  an  ohlaiind 
hy  paralltl  piojictioii.  Ilifii  tin  pair  of  IriliiKar  forms  of 
Tlifoniii  I  ndiici  to  tin  following  pair  of  biliinar  tqiia- 
hons: 

j-"(oi.i'-po-j.y-Po;j)-fo.4.r"j-'-Po.-,.r'-fo,;.rd-07(/-t-os  =  D. 
a  n  d 

y"{  J]  .V  -p  C.i/  -p  .f;;)  -p  ^4!/" J''  -p  dr,-!''  -p  di;.r  -p  d-;!)  -p  in  =  0. 
win  I  f  i:\j  =  .ij  .  j  =  1 . 4. 

Proof.  Idider  these  cotiditioiis  w<'  have  frotii  Lemma  I 
that  .4  is  aflitte  in  P-  atid  if,  =  0.  tlu'ta'fore  fd|uation  d 
reditces  to; 

163  — (aVti  p+  r'fw"  x'  —  r['  y  +  {r  I  a.]  —r[b\  p  =  0. 


Similarly.  Fquatinn  4  reduce:'  to: 

i/"(r\hn-  I'fui)'  p-j-  r'fi/"  -  t"  x'  +  (i'-!a  I  p  =  0. 

If<)th  e<|uations  are  of  tin  desired  form,  with  the  first 
four  ct>e|ficieuts  identical  across  both  e(|uaiious  [j 

;V  bilinear  function  of  three  views  has  two  advantages 
over  the  general  Irilinear  function.  First,  onl>  six  (or- 
respondin.g  points  (instead  of  nine)  aiross  thr<'e  views 
are  re<|uireil  for  sedving  for  the  coeflicirllts  Second,  the 
lower  the  degrt'e  of  the  algebraic  fmn  lion.  the  less  sen¬ 
sitive  the  .solution  shonld  Iji'  in  the  presenci'  of  errors  in 
measuring  ci.irresi>ondences.  In  other  words,  it  is  likely 
(though  tint  nece.ssarv  )  that  the  higher  order  terms,  such 
as  the  term  x".i'x  in  Fqnalion  d.  will  haw  a  higher  con¬ 
tribution  tr>  the  overall  error  sensitivity  of  the  system. 

('ompao'd  to  the  ca.se  whi'ti  all  \  •  -ws  ati  assun: -d.  nr 
thographii',  this  case  is  much  le.ss  of  an  a[iproximaiion. 
Since  the  model  views  are  taken  only  otice,  it  is  tiot  lltl- 
reasonalile  to  re(|uire  that  they  be  taken  iti  a  special 
way.  iiamelv.  with  a  tele  lens  (assuming  we  are  dealing 
with  object  recognition,  rather  than  scene  recognitioti). 
If  that  re<|uiremetit  issatislied.  then  the  recogtiition  task 
is  general  sitice  we  allow  atiy  perspective  view  to  be  taken 
during  the  rt'cogtiit ion  process. 

5  Experimental  Data 

FIk'  experimetits  dt'scrilied  itt  this  sectioti  were  dotie  itt 
order  to  evaluate  the  practical  asporl  of  using  the  trilin- 
<'ar  result  for  re-projection  compared  to  using  ejiipolar 
ititersection  and  the  litiear  combination  result  of  [ill]  (the 
latter  we  have  shown  is  .sim|))y  a  limiting  ca.se  of  the  tri- 
litiear  result ). 

File  I'pipolar  intersect ioti  iiK'thod  was  implemeuti'd  in 
the  followitig  way.  Let  /jg  atid  /d,)  be  tin'  matrices  ("es¬ 
sential"  matrices  itt  classical  termitiology  [Ipij.  which  vvt' 
aviopi  here)  that  satisfy  p” FyiP  =  0.  and  p" FrM''  —  b- 
Then,  by  incidence  of  p"  with  its  epijiolar  line,  we  have: 

p  :::!  I- y.ip  X  I- jlip  . 

Therefore,  given  eight  corri'sjionditig  jioints  across  the 
three  views,  we  cati  recover  the  two  essential  matrices, 
and  tin'll  re-jiroji'ct  all  other  object  jioiuts  otito  the  third 
view.  In  jiractice  one  would  use  more  than  eight  jioitits 
for  recovering  the  ('sseutial  matrices  in  a  litiear  or  non¬ 
linear  sijuares  method.  Siiici'  litiear  least  sejuares  meth- 
o<ls  are  still  sensitive  to  image  noise,  we  used  the  imjile- 
meiitationof  a  tton-linear  tiiethod  described  itt  [19]  which 
was  kitidly  provided  by  F.  Liiotig  atid  L.  Quaii. 

The  first  exjx'rinient  is  with  sitnulatioii  data  showing 
that  even  when  t  he  ejiipolar  geottietry  is  recovert'd  accu¬ 
rately,  it  is  still  sigtiificantly  Ix'tter  to  use  the  trilitiear 
result  which  avoids  the  jirocess  of  line  intersection.  1  he 
second  exjieritiient  is  done  on  a  real  set  of  imagi's.  cotii- 
pariiig  the  jierfortnatici'  of  the  various  methovls  and  the 
number  of  corresjiotiditig  jioints  that  are  needl'd  in  jirac- 
tice  to  achieve  reasotialile  re-jiro ject ion  results. 

5.1  Computer  Siimilatioiis 

We  used  ati  object  of  4(i  jioitits  jilaced  randomly  with  : 
coordinates  between  190  units  and  120  units,  and  x.  i/ 
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Figure  1 :  ( 'oiiipariiig  the  performance  of  I  he  epi  polar  intersect  ion  met  hod  (t  lie  dot  ted  line)  and  t  he  i  ri  linear  funct  ituis 
method  (dashetl  line)  in  the  |>resence  of  image  noise.  The  graph  on  the  left  shows  the  maximal  re-projection  ern^r 
averaged  over  '200  trials  per  noise  level  (hats  repre.sent  standard  <le\iation).  (iraph  on  the  right  dis|)lays  the  average 
re  |)i'ojection  error  averaged  over  all  re-projected  points  averaged  over  the  2()()  trials  per  noi.se  level. 


(■(vordinatf's  ranging  randomly  hi'tvveen  -12.")  and  -f  12"). 
Focal  length  was  of  ")0  units  and  the  first  view  was  oh- 
taiiK'd  hy  'I'he  second  view  (ly. )  was  gener¬ 

ated  hy  a  rotation  around  the  ()oint  (0.0.  100)  with  axis 
(0.14,0.7.0.7)  and  hy  an  angle  of  0.2  radians.  The  tliird 
view  (r.i)  was  generated  hy  a  rotation  around  an  axis 
(0,1.0)  with  the  same  translatioti  and  angle.  Various 
amounts  of  randotn  noise  was  applit'd  to  all  points  tltat 
were  to  he  re-/)rojected  onto  a  third  view,  but  not  to  the 
eight  or  nine  poitits  that  were  used  for  recovering  the 
[larametf'rs  (essential  matrices,  or  trilinear  coefficients). 
The  noise  was  random,  added  separately  to  each  coor¬ 
dinate  and  with  varying  levels  from  ().•")  to  2..")  pixel  er¬ 
ror.  We  have  done  1000  trials  as  follows:  20  random 
objects  were  created,  and  for  each  degree  of  error  the 
simulation  was  ran  10  titnes  per  object.  We  collected 
the  maximal  re-projection  error  (in  pixels)  and  the  av¬ 
erage  re-projection  error  (averaged  of  all  the  points  that 
were  re-projected).  These  numbers  were  collecte<l  sepa¬ 
rately  for  each  degree  of  error  by  averaging  over  all  trials 
(200  of  them)  and  recording  tlie  standard  deviation  as 
well.  Since  no  error  were  added  to  the  eight  or  nine 
points  that  were  used  to  determine  the  epipolar  geom¬ 
etry  and  the  trilinear  coefficients,  we  simply  solved  the 
associated  linear  systems  of  equations  required  to  obtain 
the  e.ssential  matrices  or  the  trilinear  coefficients. 

The  results  are  shown  in  Figure  1.  The  grapli  on 
the  left  shows  the  |)erformance  of  both  algorithms  for 
each  level  of  image  noise  by  measuring  the  maximal  re¬ 
projection  error.  We  see  that  under  all  noise  levels,  the 
trilinear  method  is  significantly  better  and  also  has  a 
smaller  standard  deviation.  Similarly  for  the  average  re¬ 
projection  error  shown  in  the  graph  on  the  right. 

This  difference  in  iierformance  is  ex()ected.  as  the  tri¬ 
linear  method  takes  all  three  views  together,  rather  than 
every  pair  separately,  and  thus  avoiding  line  intersec¬ 
tions. 


5.2  Experiments  On  Real  Imag«^s 

Figure  2  shows  three  views  of  the  object  we  selected  for 
the  ex|)erinienl .  The  object  is  a  s()orts  shoe  with  added 
texture  to  facilitati'  thi'  corres|)ondence  proct'ss,  I'his 
object  was  cho.sen  because  of  its  complexity,  i.e..  it  has  a 
shape  of  a  nat  ural  object  and  cannot  easily  be  described 
|)arametrically  (as  a  collection  of  planes  or  algebraic  sur¬ 
faces).  .Note  that  the  situation  de|)icted  here  is  challeng¬ 
ing  because  the  re-projected  view  is  not  in-b<‘tween  the 
two  model  views,  i.e..  one  should  expect  a  larger  sensi¬ 
tivity  to  image  noise  than  in-between  situations.  A  set  of 
24  points  were  manually  selected  on  one  of  the  frames, 
I'l,  and  their  correspondences  were  automatically  ob¬ 
tained  along  all  other  frames  used  in  this  I'xperiment. 
The  correspondence  process  is  based  on  an  imphuiienta- 
tion  of  a  coarse-to-fine  optical-flow  algorithm  described 
in  [7],  To  achieve  accurate  correspondences  across  dis¬ 
tant  views,  intermediate  in-bet  ween  frames  were  taken 
and  the  dis|)lacements  across  consecutive  frames  w<‘re 
added.  'File  overall  disjilacement  fhdd  was  then  used  to 
push  ("warp")  the  first  frame  towards  the  target  frame 
and  thus  create  a  synthetic  image.  Oi)tical-flow  was  a|)- 
plied  again  between  the  synthetic  frame  and  the  targi't 
frame  and  the  resulting  displacement  was  added  to  the 
overall  displacement  obtained  earlier.  This  i)roce.ss  pro¬ 
vides  a  dense  displacenn'iit  field  which  is  then  sam()hHl 
to  obtain  the  correspondences  of  the  24  points  initially 
chosen  in  the  first  frame.  The  results  of  this  process  an' 
shown  in  Figure  2  by  displaying  .S(|uares  centered  arouinl 
the  com[)uted  locations  of  the  corresponding  |)oints.  One 
can  see  that  the  correspondences  obtaiiu'd  in  this  manner 
are  rea,sonable.  and  in  most  cases  to  sub-pixel  accuracy. 
One  can  readily  automate  further  this  ()rocess  lyv  select¬ 
ing  points  in  the  first  fram*'  for  which  tlu'  H<'ssian  ma¬ 
trix  of  spatial  derivatives  is  well  conditioned  similar 
to  the  confidence  values  suggest('d  in  the  im[)|ementa- 
tions  of  [4.  7.  20]  -  howevf'r,  the  int('ntion  here  was  not 
,  so  much  as  to  hnild  a  com[)l(‘te  syst('m  but  to  test  the 
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Figure  2:  Top  How:  Two  model  views,  i  i  on  tlie  left  and  t-j  on  the  right.  The  overlayed  .s(|uares  illustrate  the 
corresponding  points  (31  i)oints).  Bolioto  Row:  Third  view  i;,.  .Note  that  13  is  not  in-hetween  1.  1  and  i  ■_).  making 
the  re-projection  problem  more  challenging  (i.e..  performance  is  more  sensitive  to  image  noise  than  in-hetween 
situations). 


Figure  3:  Re-projection  onto  I’a  using  the  trilinear  result.  'I'he  re-projected  points  are  marked  as  crosses.  thereror<> 
should  he  at  the  center  of  the  scpiares  for  accnrate  re-projection.  On  the  left,  the  minimal  numher  of  points  were  used 
for  recovering  the  trilinear  coefficients  (nine  points);  the  average  pi.xel  error  between  the  true  an  estimated  locations 
is  1.4,  and  the  maximal  error  is  5.7.  On  the  right  12  points  were  used  in  a  least  .scpiares  fit;  average  error  is  0.1  and 
maximal  error  is  1.4. 

I) 


Figure  1:  liesults  of  re-projertioii  using  iulerst-ct ion  of  epipolar  lines.  I  Ih-  |■e-))roje^l<■(l  points  aii-  niarkisl  as  crossi  s, 
tlit'refore  slioultl  lie  at  the  cetiter  of  the  stptares  for  accurate  re-pro j<-ct iott.  In  the  lefthtitul  ilisplay  the  grouiicl  plane 
jioints  were  used  for  recoveritig  the  essetitial  tiiatrix  (se»'  text),  and  in  the  righthatid  display  the  essential  tiiatrices 
were  recovered  front  the  itn|)lemeutatioti  of  [19]  using  all  ;M  points  across  the  three  views.  Maxiniittti  displaci  nient 
error  itt  the  lefthatid  display  is  2-’).7  pixels  atid  average  error  is  7.7  pixels  .\Iaxittial  I'rror  in  the  righthand  displa\  is 
FI.  I  pixels  ainl  average  error  is  9. ■'it'  pixels. 


perforinattce  of  the  trilitiear  re-project  iott  tnethod  and 
cotniiare  it  to  the  perfortiiatice  of  c'pipolar  intersection 
and  the  linear  cotnhitiatioti  itiethods. 

I'he  trilitiear  tnethod  reriuires  at  least  tiitie  corri'siiond- 
ing  points  across  the  three  views  (we  tieed  17  equation, 
aitil  nitie  poitils  provide  18  ('cptatiotis).  whereas  epipolar 
intersection  catt  he  done  (in  jirinciph')  with  eight  points. 
The  cpiestioti  we  are  ahoitt  to  address  is  what  is  the 
tutiither  of  poittts  that  are  reciuired  itt  practice  (due  to 
errors  iti  correspotidetice.  letts  rlistortiotts  atid  other  *'f- 
fects  that  are  tiot  adecptately  modeled  hy  the  pin-hole 
camera  tiiodel)  to  achieve  rea.sottahle  perfortiiatice? 

The  trilitiear  result  was  first  applied  with  the  tiiinimal 
iiumher  of  points  (tiitie)  for  solviitg  for  the  coefficients, 
and  theti  applied  with  12  poitits  itsitig  a  litiear  least- 
sipiares  solutioti.  The  results  an-  shown  iti  Figure  d. 
.Nine  poitits  provide  a  r<“-projection  with  maximal  error 
of  .7.7  pi.xels  and  average  error  of  1 .4  pixels.  4  he  solution 
using  12  points  provided  a  significant  improvement  with 
maximal  error  of  1.4  and  averagi'  error  of  0.4  pi.xels.  I's- 
ing  more  points  did  not  improve  significantly  the  results; 
for  exaniiile.  when  all  ;54  points  were  used  the  maximal 
error  went  down  to  1.14  pixels  and  average  error  stayed 
at  0.42  pi.xels. 

.Next  the  epipolar  intersection  method  was  afiplied. 
We  used  two  methods  for  recovering  the  essential  matri¬ 
ces.  One  method  is  by  using  the  imiilementation  of  [19]. 
and  the  other  is  by  taking  advantage  that  four  of  the  cor¬ 
responding  points  are  coming  from  a  plane  (the  ground 
plane).  In  the  former  ca.se.  much  more  than  eight  points 
were  reipiired  in  order  to  achieve  rea.sonable  results.  For 
example,  when  using  all  the  111  [loints.  the  maximal  er¬ 
ror  was  4;1.4  pixels  ami  the  average  error  was  9.58  pixels. 
In  the  latter  case,  we  recovered  first  the  homography  1) 
due  to  the  ground  jilaiie  and  then  the  ppi|)ole  c"  using 
two  additional  points  (those  on  the  film  cartridges).  It 


is  then  known  (see  [2(1.  20])  that  /  j,-,  =  where  [e"] 

is  the  anti-symmetric  matrix  of  c".  A  similar  procedure 
was  used  to  recover  /•]>:)  I  lierefore.  only  six  points  were 
used  for  re-|>roiection.  but  nevertheless,  the  results  were 
slightly  better:  maximal  error  of  25.7  pixids  and  average 
('rror  of  7.7  pixt'ls.  Figure  4  shows  these  results. 

I'inally.  we  tested  tin'  iierformauce  of  re-|)roj('ctiou  us- 
ingthe  linear  combination  method.  Since  the  linear  com¬ 
bination  methods  holds  oidy  for  orthographic  views,  we 
are  actually  testing  the  ort hogra|)hic  assumption  under 
a  perspective  situation,  or  in  other  words,  whethi'r  thi' 
higher  (bilinear  and  Irilinear)  order  terms  of  the  trilin- 
ear  e(|uations  are  significant  or  not.  Fhe  linear  combina¬ 
tion  method  recjuires  at  least  four  corresponding  |>oints 
across  the  three  views.  We  aiijdied  the  method  with  four. 
12  (for  com|>arison  with  the  trilitiear  cast'  shown  in  Fig¬ 
ure  ;l).  and  all  111  points  (the  lattf'r  two  using  linear  h'ast 
stpiares).  I'he  results  are  tiisplayed  in  F'igun'  5.  Fhe  pi'r- 
formance  in  all  cast's  are  significantly  ptiorer  than  when 
using  the  trilitiear  functions,  but  better  than  t  lit' epipolar 
intersect  ion  methinl. 

6  Discussion 

We  have  .sf'en  tliat  any  vit'w  of  a  fixetl  III)  object  can 
be  expressetl  as  a  trilint'ar  function  with  two  rt'fert'iice 
views  in  tilt'  geiit'ral  case,  or  as  a  biliiit'ar  function  wlit'ii 
lilt'  reference  vit'ws  are  crealeil  by  means  of  parallel  pro¬ 
jection.  Thest'  functions  jirovitle  allt'riial ive.  much  sim- 
pli'r.  means  for  iiiauipiilaliiig  views  of  a  sct'iit'  han  ot  her 
methods.  lix(>t'rimental  results  show  that  the  trilint'ar 
functions  are  also  ii.sefiil  in  practice  yielding  |>erformance 
that  is  significantly  bt'ller  than  efripolar  intt'rseci ion  or 
the  lint'ar  combiiiat ion  iiit'thotl. 

Tilt'  application  that  was  em|)hasizeil  throughcut  the 
paper  is  visual  rt'cognition  via  alignmt'iit .  Heasonabh' 


Figllr^'  5:  fiesults  of  re-projoctioii  using  the  lin  ar  roiiiUiiiatioii  of  views  iiietliod  |)ioi)ose(l  hy  [;M]  (appli<al)lf  to 
j)aral|p|  j)rojerlion).  Top  Row:  In  the  left  hand  display  the  linear  coeffirients  were  recovered  from  four  corresponding 
points;  maxitiial  error  is  .j().7  pixels  and  average  error  is  20. d  pixels.  In  the  righthand  disi)lay  the  coeffici<'nts  were 
recovered  using  12  points  in  a  linear  least  stpiares  fashion:  maximal  error  is  2d.d  i>ixels  and  average  error  is  O.is  pixels. 
Bottom  Row:  The  coefficients  were  recovered  u.sing  all  CM  |>oints  across  the  three  views.  Maximal  error  is  20. -1  pixels 
and  average  error  is  -o.Ud  juxels. 
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|»  rroriiiaii(f  \va>  i  lilaiin  d  with  I'J  <iiri>-^l»iii  luiu,  |>i>iiii!' 
wit  ii  I  III'  111  arl  \  ii'W  ( I  1 1  whirl  I  iiia\  In  im.  many  11  l  In 
imaur  III  liiiiili'l  mat rliiim  is  ilmn  hy  living  all  |iiis?.ili|i 

II  niilniial  mils  1 1|  |mml  mat  r  I  ns  1  In  ixisl  iini  i  if  lijljinar 
Imirl  lulls  III  till  s|ii(ial  rasi  ulnri  tin  iinnlil  is  mlln*- 
,i;ia|ilili'.  hill  tin  iidmI  vnw  is  [n  |■>|ll■l■|  im-,  is  nniii  i-n- 
i  niiraniim  Irum  tin  siaiiil|iiimi  i >1  1 1 iiiiit iii"  initiits  Ih  ii' 
wr  lia\i  iln  ri'siill  that  iiiil\  sis  i  iitri'>|iiiinliiin  |>iiiiils 
ari'  ri  (|mi'i'i|  in  uhlain  ii  rimiiilinii  i if  j)i'isj)i  r( ivi-  vii  ws 
( pii  i\  nil'll  wi  ran  satisfy  llir  ri'i|nii-i‘mi'iil  that  tin  nmili  I 
is  '  ii'i  In  ij;ia|iliir  |.  \\i  lia\i  in>l  i'X|ii  liiiii'iili  il  with  hilm- 
lar  Imirliiiiis  111  si'i  Iniw  maii\  imiiils  wmilil  In  iiiiiliil 

III  |irarliri'.  hill  plan  Iniln  llial  m  lln  fullin'  tiiraiisi 
Ilf  llii'ir  simplirit\.  mn  ma\  spirulali'  llial  llnsi'  al;;i 
hrair  fiiiiri  mils  will  liml  iisis  m  lasks  nllni'  lliaii  \isiial 
I'l'i'i  in  ml  mn  soiin' nf  ilnisr  arr  liisriissi'il  In  lnw. 

1  lii'i'i  ma\  I'xisi  iillnr  appliral  ii ms  wlmri  simpliril> 
is  Ilf  majiir  impiirlanri'.  wln  ii  as  tin'  iiiimln  r  nf  pnmi', 
is  li'ss  111  a  riiiiri'i'ii.  (  1  iiisiili  i'  Ini'  i'xam|i|i  .  lln-  appli- 
ralimi  III  nil nli'l-hasi'il  riimpi'i'ssinn.  With  lln  inliinai' 
liilirt  il  Ills  wi  ni  l'll  17  paraiin  ii'i's  in  ri  pri  si  iii  a  vmw  as 
a  fmirlinii  nf  Iwn  ri  fi  ii  iiri'  \  ii'Ws  in  full  ri ii  rrspniiili  lirt'. 
Assiimi'  hntli  till'  si'iiili'i'  and  tlir  riri  iM-r  liavi-  lln-  Iwn 
|■|'l^'n■nrl'  virws  and  apply  I  In'  saiiii'  afnnrii  Inn  fnr  nhlain- 
inj!,  rnirispninli'liii's  hilwiin  till'  twn  virWs.  In  sriid 
a  lliil'd  vii'W  (iniinrinv,  plnhli  llis  I  ,  si  lf  nrrlllsinns  that 
nmlil  III'  dralt  srparaii  ly )  I  In'  si  ndri'  ran  snlvr  fnr  tin 
17  |iarami'li'is  iisinji  man>  pnints.  hut  I'M  iitnally  si-inl 
only  till'  17  paranii'li  rs.  l  ln'  iin  iv i  r  tln  n  simply  rnm- 
hiiii's  till'  Iwn  n'(l■l'l'nrl'  vii  Ws  in  a  "iriliin  ai'  way’  "ivi  ii 
till'  |■l'('l'iv^d  pai'aini'li'i's.  I  his  is  i  h  arly  a  domain  wlii-i'i' 
till'  iimiihi'i'  of  points  ail'  imt  a  major  rnni  i'i'ii,  whi-ri'.is 
siiiiplicit  j ,  and  rohnstin  ss  (as  shown  ahovi')  dn<'  In  ihr 
shorl-riil  in  tlm  rnmpiilatinns.  isnfiiiiat  imporlanri'. 

Hi'lati'd  In  iiiiaui' rodim;.  an  approach  of  imaji,i' di'rom- 
posilion  into  "layi'i's"  was  ri'cinlly  proposed  hy  [l.’i].  In 
this  approach,  a  si'i|ni’nri'  of  views  is  divided  up  into  re- 
ifions.  whose  motion  of  earh  is  desrrihed  ap|iro.ximate|y 
hy  a  "ill  afliiie  transformation.  1  he  sender  sends  the  lirst 
iniaj;e  followed  only  hy  the  six  altine  paranielers  for  each 
rejiion  for  each  siihse(|nent  frame.  I'he  use  of  al};ehrair 
fnnrtions  o|  views  can  potent  iaily  make  this  approach 
more  powerful  heranse  instead  of  (lividin;!;  up  the  .scene 
into  planes  (il  would  have  heen  planes  if  the  projeciion 
was  |)arallel.  in  general  its  not  even  planes)  imi  can  at¬ 
tempt  to  divide  the  scene  into  ohjerls.  each  carries  the 
17  paranielers  descrihinn  its  displacement  onto  tin-  siih- 
seqnent  frame. 

.■\noiher  area  ol  application  may  he  in  computer 
jira|ihics.  He-project  ion  lechnii|nes  provide  a  short-cut 
for  imafie  renderiiif!;.  (liven  two  fully  rendereil  views 
of  some  ;i|)  ohject.  other  views  ( again  ignoring,  self- 
occliisions)  can  he  rendered  hy  simply  ■comhiniiig  tin 
reference  views.  .Again,  the  nnmher  of  ■•nriespondin;; 
jioints  is  less  of  a  con  'rn  here. 
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